Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: End-to-End Integration Test for Client-side Tracing in Firestore Java Server SDK using OpenTelemetry SDK and Cloud Trace Exporter against Cloud Trace. #1635

Merged
merged 35 commits into from
Apr 3, 2024

Conversation

jimit-j-shah
Copy link
Contributor

@jimit-j-shah jimit-j-shah commented Mar 28, 2024

test: End-to-End Integration Test for Client-side Tracing in Firestore Java Server SDK using OpenTelemetry SDK and Cloud Trace Exporter against Cloud Trace.

The test is parameterized to run all the cases with and without the use of a Global OpenTelemetry SDK instance.

In order to run these tests, the test-run user must have permissions listed in https://cloud.google.com/trace/docs/iam#trace-roles so the test can ingest into and read traces from Cloud Trace

Fixes #1604 ☕️

…race fetching using retries for missing or incomplete traces due to eventual consistency of Cloud Trace
…or Transaction ops (BeginTransaction, Rollback etc)
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. api: firestore Issues related to the googleapis/java-firestore API. labels Mar 28, 2024
@jimit-j-shah jimit-j-shah self-assigned this Mar 28, 2024
@jimit-j-shah jimit-j-shah changed the title Jimit/tracing 1 test: ITE2ETraceTest implements End-to-End Integration Test for Client-side Tracing in Firestore Java Server SDK using OpenTelemetry SDK and Cloud Trace Exporter against Cloud Trace. Mar 28, 2024
@jimit-j-shah jimit-j-shah changed the title test: ITE2ETraceTest implements End-to-End Integration Test for Client-side Tracing in Firestore Java Server SDK using OpenTelemetry SDK and Cloud Trace Exporter against Cloud Trace. test: End-to-End Integration Test for Client-side Tracing in Firestore Java Server SDK using OpenTelemetry SDK and Cloud Trace Exporter against Cloud Trace. Mar 28, 2024
@jimit-j-shah jimit-j-shah marked this pull request as ready for review March 28, 2024 20:29
@jimit-j-shah jimit-j-shah requested review from a team as code owners March 28, 2024 20:29
Copy link
Contributor

@ehsannas ehsannas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall. A few suggestions outlied below.

…shorter than the trace callstack - don't need to throw an exception in that case
@ehsannas ehsannas merged commit 84e0101 into tracing Apr 3, 2024
16 checks passed
@ehsannas ehsannas deleted the jimit/tracing-1 branch April 3, 2024 22:20
ehsannas added a commit that referenced this pull request Apr 26, 2024
feat: client-side tracing.

feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions. (#1531)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* Address code review feedback.

feat: Add com.google.cloud.firestore.telemetry package. (#1533)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* Address code review feedback.

* Factor out the otel version in pom.xml.

fix: Remove OpenCensus tracing code. (#1589)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* fix: Remove OpenCensus tracing code.

feat: tracing for aggregate queries, bulkwriter, partition queries, a… (#1590)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* fix: Remove OpenCensus tracing code.

* feat: tracing for aggregate queries, bulkwriter, partition queries, and listDocuments.

* Address code review feedback.

* Address feedback.

* don't use wildcard imports.

feat: trace instrumentation for DocumentReference methods. (#1591)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* fix: Remove OpenCensus tracing code.

* feat: tracing for aggregate queries, bulkwriter, partition queries, and listDocuments.

* feat: trace instrumentation for DocumentReference methods.

feat: trace instrumentation for queries and transactions. (#1592)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* fix: Remove OpenCensus tracing code.

* feat: tracing for aggregate queries, bulkwriter, partition queries, and listDocuments.

* feat: trace instrumentation for DocumentReference methods.

* feat: trace instrumentation for queries and transactions.

* test: Adding first e2e client-tracing test w/ Custom Root Span (#1621)

* test: Adding first e2e client-tracing test w/ Custom Root Span

* Roll back E2E tests commit.

* Address feedback.

* Address feedback (better event log message).

* Address feedback.

---------

Co-authored-by: Jimit J Shah <[email protected]>

test: End-to-End Integration Test for Client-side Tracing in Firestore Java Server SDK using OpenTelemetry SDK and Cloud Trace Exporter against Cloud Trace. (#1635)

* Adding first e2e client-tracing test w/ Custom Root Span

* test: Adding first e2e client-tracing test w/ Custom Root Span

* Fixing test dependencies and use default GCP testing project.

Fixing

* Fixing test dependencies and use default GCP testing project.

* Fixing formatting

* Add aggregationQueryGet Test

* Add bulkWriterCommitTrace Test

* Fixing running multiple-tests

* Add partitionQuery Test

* Add collectionListDocumentsTrace Test

* Add docRef*Trace Tests

* Add docRefUpdate*Trace and docRefDelete*Trace Tests

* Fixing Trace fetching using retries for missing or incomplete traces due to eventual consistency of Cloud Trace

* Add get/query Trace Tests

* Add Transaction test

* Added TraceContainer to be able to test transaction test-cases

* test: Adding Transaction tests

* test: Adding Transaction tests

* test: Adding TestParameterInjector to run the test for global and non-global opentelemetry SDK instances

* test: formatting and cleanup

* test: Adding first e2e client-tracing test w/ Custom Root Span

* test: Add aggregationQueryGet Test

* test: Add bulkWriterCommitTrace Test and fixed running multiple-tests

* test: Add partitionQuery Test

* test: Add collectionListDocumentsTrace Test

* test: Add docRefUpdate*Trace and docRefDelete*Trace Tests and fixed Trace fetching using retries for missing or incomplete traces due to eventual consistency of Cloud Trace

* test: Add get/query Trace Tests

* test: Added Transaction tests using TraceContainer to verify traces for Transaction ops (BeginTransaction, Rollback etc)

* test: Adding TestParameterInjector to run the test for global and non-global opentelemetry SDK instances

* test: Formatting and cleanup

* test: review comments

* test: fixing dfs to handle case where the compareTo callstack may be shorter than the trace callstack - don't need to throw an exception in that case

* test: Consolidating verification methods

* test: review comments

fix: Make telemetry-related fields transient. (#1638)

fix: Rename 'enabled' to 'tracingEnabled'. (#1639)

* fix: Rename 'enabled' to 'tracingEnabled'.

In the future, FirestoreOpenTelemetryOptions will support enabling/disabling
Logging and Metrics as well. So we should use a better name for this field.

* address feedback.
ehsannas added a commit that referenced this pull request Jul 18, 2024
* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions. (#1531)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* Address code review feedback.

* feat: Add com.google.cloud.firestore.telemetry package. (#1533)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* Address code review feedback.

* Factor out the otel version in pom.xml.

* fix: Remove OpenCensus tracing code. (#1589)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* fix: Remove OpenCensus tracing code.

* feat: tracing for aggregate queries, bulkwriter, partition queries, a… (#1590)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* fix: Remove OpenCensus tracing code.

* feat: tracing for aggregate queries, bulkwriter, partition queries, and listDocuments.

* Address code review feedback.

* Address feedback.

* don't use wildcard imports.

* feat: trace instrumentation for DocumentReference methods. (#1591)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* fix: Remove OpenCensus tracing code.

* feat: tracing for aggregate queries, bulkwriter, partition queries, and listDocuments.

* feat: trace instrumentation for DocumentReference methods.

* feat: trace instrumentation for queries and transactions. (#1592)

* feat: Add FirestoreOpenTelemetryOptions to FirestoreOptions.

* feat: Add com.google.cloud.firestore.telemetry package.

* fix: Remove OpenCensus tracing code.

* feat: tracing for aggregate queries, bulkwriter, partition queries, and listDocuments.

* feat: trace instrumentation for DocumentReference methods.

* feat: trace instrumentation for queries and transactions.

* test: Adding first e2e client-tracing test w/ Custom Root Span (#1621)

* test: Adding first e2e client-tracing test w/ Custom Root Span

* Roll back E2E tests commit.

* Address feedback.

* Address feedback (better event log message).

* Address feedback.

---------

Co-authored-by: Jimit J Shah <[email protected]>

* test: End-to-End Integration Test for Client-side Tracing in Firestore Java Server SDK using OpenTelemetry SDK and Cloud Trace Exporter against Cloud Trace. (#1635)

* Adding first e2e client-tracing test w/ Custom Root Span

* test: Adding first e2e client-tracing test w/ Custom Root Span

* Fixing test dependencies and use default GCP testing project.

Fixing

* Fixing test dependencies and use default GCP testing project.

* Fixing formatting

* Add aggregationQueryGet Test

* Add bulkWriterCommitTrace Test

* Fixing running multiple-tests

* Add partitionQuery Test

* Add collectionListDocumentsTrace Test

* Add docRef*Trace Tests

* Add docRefUpdate*Trace and docRefDelete*Trace Tests

* Fixing Trace fetching using retries for missing or incomplete traces due to eventual consistency of Cloud Trace

* Add get/query Trace Tests

* Add Transaction test

* Added TraceContainer to be able to test transaction test-cases

* test: Adding Transaction tests

* test: Adding Transaction tests

* test: Adding TestParameterInjector to run the test for global and non-global opentelemetry SDK instances

* test: formatting and cleanup

* test: Adding first e2e client-tracing test w/ Custom Root Span

* test: Add aggregationQueryGet Test

* test: Add bulkWriterCommitTrace Test and fixed running multiple-tests

* test: Add partitionQuery Test

* test: Add collectionListDocumentsTrace Test

* test: Add docRefUpdate*Trace and docRefDelete*Trace Tests and fixed Trace fetching using retries for missing or incomplete traces due to eventual consistency of Cloud Trace

* test: Add get/query Trace Tests

* test: Added Transaction tests using TraceContainer to verify traces for Transaction ops (BeginTransaction, Rollback etc)

* test: Adding TestParameterInjector to run the test for global and non-global opentelemetry SDK instances

* test: Formatting and cleanup

* test: review comments

* test: fixing dfs to handle case where the compareTo callstack may be shorter than the trace callstack - don't need to throw an exception in that case

* test: Consolidating verification methods

* test: review comments

* fix: Make telemetry-related fields transient. (#1638)

* fix: Rename 'enabled' to 'tracingEnabled'. (#1639)

* fix: Rename 'enabled' to 'tracingEnabled'.

In the future, FirestoreOpenTelemetryOptions will support enabling/disabling
Logging and Metrics as well. So we should use a better name for this field.

* address feedback.

* fix: Minor improvement to the ITE2ETracingTest. (#1637)

* fix: Minor improvement to the ITE2ETracingTest.

* revert the numExpectedSpans change.

* feat: Add 'isTransactional' attribute. (#1657)

* fix: Necessary test improvements for CI environments. (#1673)

* fix: Necessary test improvements for CI environments.

* Address feedback.

* feat: Disable the tracing feature and remove public APIs.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Add the Firestore SDK version to the attributes.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Update the "test" dependency versions.

* Address feedback related to attributes.

* Add 'project_id' attribute.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Revert a736fcc ("Disable the tracing feature and remove public APIs").

* GlobalOtel reset for test must happen in `before`, not `after`.

* Address feedback.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Jimit J Shah <[email protected]>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: firestore Issues related to the googleapis/java-firestore API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

E2E Tests for Client-Side Tracing using OpenTelemetry and Cloud Trace Exporter
2 participants